Disentangling correlation between speed and ability at the subject level and between intensity and difficulty at the item level from psycholinguistic data: a joint modeling approach

In psycholinguistic experiments multiple subjects are faced with multiple test items. Despite the early 70's paper of Clark (1973) arguing that averaging reaction times from such experiments over items for each subject and averaging over subjects for each item respectively, using these means in ANOVA-models (referred to as F1 and F2 statistics), and drawing inference from both statistics separately may be incorrect, the vast majority of published psycholinguistic results employed such techniques over the last decades. Baayen et al. (2008) explained in detail a mixed effects modeling approach with crossed random effects for subjects and items. In addition to the reaction times, psycholinguistic literature is often describing accuracy as well. Accuracy is then summarized by simple frequency tables exploiting the binary outcomes (i.e. correct or incorrect response) measured for each subject-item combination. To improve on this and allow for estimation of covariates effects on the accuracy, Jaeger (2008) introduced in the psycholinguistic literature a model for the probability of a correct answer. More specifically, he proposed a mixed logistic regression model that allows for crossed random subject and item effects along the lines of Baayen et al. (2008). Unfortunately, reaction times and accuracy are most often described separately without any concern being raised about their correlation. It is important to get a better understanding of the correlation between reaction times and accuracy, if any. The natural next step is therefore to consider a joint model for the reaction time and the accuracy. Joint modeling of these 2 outcomes can most easily be performed in a hierarchical framework. Van der Linden (2007) proposed an item-response theory model, a model for response time distribution and a higher-level structure accounting for the dependencies between the item and subjects parameters in these models. His hierachical framework is very exible in that any item-response or response time model can be substituted. Building on Van der Linden's work, we first provide a framework that combines the models introduced in the psycholinguistic literature by Baayen et al. (2008) and Jaeger (2008), treats subjects and items as random, and allows for correlation between reaction time and accuracy. The main advantage of this framework is its ability to disentangle between correlation driven by subjects and correlation driven by items. Estimation of the model parameters in the joint model and model checking are performed in a Bayesian approach with Markov Chain Monte Carlo (MCMC). The performance of the proposed methodology is illustrated with a real-data example

The analysis of correlated non-Gaussian outcomes from clusters of size two: non-multilevel-based alternatives?

In this presentation we discuss the analysis of clustered binary or count data, when the cluster size is two. For Gaussian outcomes, linear mixed models taking into account the correlation within clusters, are frequently used and well understood. Here we explore the potential of generalized linear mixed models (GLMMs) for the analysis of non-Gaussian outcomes that are possibly negatively correlated. Several approximation techniques (Gaussian quadrature, Laplace approximation or linearization) that are available in standard software packages for these GLMMs are investigated. Despite the different modelling options related to these different techniques, none of these have satisfactory performance in estimating fixed effects when the within-cluster correlation is negative and/or the number of clusters is relatively small. In contrast, a generalized estimating equations (GEE) approach for the analysis of non-Gaussian data turns out to have an overall excellent performance. When using GEE the robust score and Wald test are recommended for small and large samples, respectively

Evaluating of bootstrap procedures for fMRI data

Over the last decade the bootstrap procedure is gaining popularity in the statistical analysis of neuroimaging data. This powerful procedure can be used for example in the non-parametric analysis of neuro-imaging data. As fMRI data are complexly structured with both temporal and spatial dependencies, such bootstrap procedures may indeed offer an elegant solution. However, a thorough investigation on the most appropriate bootstrapping procedure for fMRI data has to our knowledge never been performed. Friman and Westin (2005) showed that a bootstrap procedure based on pre-whitening the temporal structure of fMRI time series is superior to procedures based on wavelets or Fourier decomposition of the signal, especially in the case of blocked fMRI designs. For time-series, several bootstrap schemes can be exploited though (see e.g. Lahiri, 2003). For the re-sampling of residuals from general linear models fitted on fMRI data, we examine more specifically here the differences between 1) bootstrapping pre-whitened residuals which are based on parametric assumptions of the temporal structure, 2) a blocked bootstrapping which avoids making such assumptions (with several variants like the circular bootstrap,. . . ), and 3) a combination of both bootstrap procedures. We furthermore explore whether the bootstrap procedures is best applied before or after smoothing the volume of interest. Based on real data and simulation studies, we discuss the temporal and spatial properties of the bootstrapped volumes for all possible combinations and nd interesting differences

Evaluation of second-level inference in fMRI analysis

We investigate the impact of decisions in the second-level (i.e., over subjects) inferential process in functional magnetic resonance imaging on (1) the balance between false positives and false negatives and on (2) the data-analytical stability, both proxies for the reproducibility of results. Second-level analysis based on a mass univariate approach typically consists of 3 phases. First, one proceeds via a general linear model for a test image that consists of pooled information from different subjects. We evaluate models that take into account first-level (within-subjects) variability and models that do not take into account this variability. Second, one proceeds via inference based on parametrical assumptions or via permutation-based inference. Third, we evaluate 3 commonly used procedures to address the multiple testing problem: familywise error rate correction, False Discovery Rate (FDR) correction, and a two-step procedure with minimal cluster size. Based on a simulation study and real data we find that the two-step procedure with minimal cluster size results in most stable results, followed by the familywise error rate correction. The FDR results in most variable results, for both permutation-based inference and parametrical inference. Modeling the subject-specific variability yields a better balance between false positives and false negatives when using parametric inference

A review of R-packages for random-intercept probit regression in small clusters

Generalized Linear Mixed Models (GLMMs) are widely used to model clustered categorical outcomes. To tackle the intractable integration over the random effects distributions, several approximation approaches have been developed for likelihood-based inference. As these seldom yield satisfactory results when analyzing binary outcomes from small clusters, estimation within the Structural Equation Modeling (SEM) framework is proposed as an alternative. We compare the performance of R-packages for random-intercept probit regression relying on: the Laplace approximation, adaptive Gaussian quadrature (AGQ), Penalized Quasi-Likelihood (PQL), an MCMC-implementation, and integrated nested Laplace approximation within the GLMM-framework, and a robust diagonally weighted least squares estimation within the SEM-framework. In terms of bias for the fixed and random effect estimators, SEM usually performs best for cluster size two, while AGQ prevails in terms of precision (mainly because of SEM's robust standard errors). As the cluster size increases, however, AGQ becomes the best choice for both bias and precision

Cross-linguistic activation in bilingual sentence processing: the role of word class meaning

This study investigates how categorial (word class) semantics influences cross-linguistic interactions when reading in L2. Previous homograph studies paid little attention to the possible influence of different word classes in the stimulus material on cross-linguistic activation. The present study examines the word recognition performance of Dutch-English bilinguals who performed a lexical decision task to word targets appearing in a sentence. To determine the influence of word class meaning, the critical words either showed a word class overlap (e. g. the homograph tree [ noun], which means "step" in Dutch) or not (e.g. big [ADJ], which is a noun in Dutch meaning "piglet"). In the condition of word class overlap, a facilitation effect was observed, suggesting that both languages were active. When there was no word class overlap, the facilitation effect disappeared. This result suggests that categorial meaning affects the word recognition process of bilinguals

A new approach for within-subject mediation analysis in AB/BA crossover designs

Crossover trials are widely used in psychological and medical research to assess the effect of reversible exposures. In such designs, each subject is randomly allocated to a sequence of conditions, enabling the evaluation of treatment differences within each individual. When there are but two possible exposures -each assessed during one of two time periods-, the crossover study is referred to as an AB/BA design. The goal of this presentation is to discuss mediation analysis in such simple crossover studies. We do so by considering within-subject mediation from a counterfactual-based perspective and by deriving expressions for the direct and indirect effects. Employing simulation studies, the performance of several existing methods will be assessed and compared to that of a novel one we propose. We show that the new method yields unbiased and efficient estimators for the direct and indirect effect, under a minimalistic set of `no unmeasured confounding'-assumptions. Finally, we illustrate the different techniques with data from a neurobehavioral study

Data-analytical stability in second-level fMRI inference

We investigate the impact of decisions in the second-level (i.e. over subjects) inferential process in functional Magnetic Resonance Imaging (fMRI) on 1) the balance between false positives and false negatives and on 2) the data-analytical stability (Qiu et al., 2006; Roels et al., 2015), both proxies for the reproducibility of results. Second-level analysis based on a mass univariate approach typically consists of 3 phases. First, one proceeds via a general linear model for a test image that consists of pooled information from different subjects (Beckmann et al., 2003). We evaluate models that take into account first-level (within-subjects) variability and models that do not take into account this variability. Second, one proceeds via permutation-based inference or via inference based on parametrical assumptions (Holmes et al., 1996). Third, we evaluate 3 commonly used procedures to address the multiple testing problem: family-wise error rate correction, false discovery rate correction and a two-step procedure with minimal cluster size (Lieberman and Cunningham, 2009; Bennett et al., 2009). Based on a simulation study and on real data we find that the two-step procedure with minimal cluster-size results in most stable results, followed by the family- wise error rate correction. The false discovery rate results in most variable results, both for permutation-based inference and parametrical inference. Modeling the subject-specific variability yields a better balance between false positives and false negatives when using parametric inference